Yahoo!'s Web Ignites the Robot Industry
By Paul J. Bruemmer


Everyone is trying to second-guess what's going on at Yahoo!. Some experts are saying webmasters will stop paying $299 yearly for the Yahoo! Business Express listing because they've seen their rankings tank since Yahoo! started serving Google crawler-based results. Others say it's more important than ever to pay for the Yahoo! listing in order to boost link popularity, an important algorithm ranking factor.

What Changed at Yahoo!?

Yahoo! used to rank sites based on it's human-powered directory database with an algorithm of its own, but filled in with Google crawler-based results when it lacked data on search terms in its own database. This meant that a Yahoo! listing with excellent title and description produced high rankings on Yahoo!.

With the recent change, Yahoo! is now using Google crawler-based results (and algorithm) in its default Web results, substituting Yahoo! directory listings wherever possible. But basically what you get is Google results. Thus, Yahoo! rankings can change substantially for better or for worse.

Yahoo still displays its directory listings, but users must click on a Web Site Directory link (below the fold) to see these results, and most people don't do this because it's easier to use the default search box. So not many users are going to see your Yahoo! Web Site Directory listing.

When these changes were first announced, some webmasters and SEO experts questioned the value of a Yahoo! listing at $299 per year. But knowing that a good portion of its revenues come from the Business Express listings, would Yahoo! jeopardize this source of income? Not likely.

First of all, Yahoo! left open the possibility of pulling results from other databases such as Inktomi. Then, it knows that link popularity is heavily weighted in many crawler algorithms besides Google, making the Yahoo! listing advantageous to boost chances of a top-ten listing in Google and other crawler-powered databases like AltaVista, AOL, Fast, Lycos, HotBot, MSN, Teoma, etc.

Opinion of Industry Experts

Danny Sullivan, Internet consultant, author, and renown search engine expert, concluded that the Yahoo! listing is as important as ever.

"The value of Business Express is no longer essential for just small business, but for big business it still remains a no-brainer purchase," said Sullivan. "Because Yahoo! is making use of spider-based listings more than they ever did in the past, it is more valuable than ever to optimize your pages. These changes make it more important that companies consider their search engine strategy."

Because of the Yahoo! changes, iProspect CEO Fredrick Marckini believes "This changes everything: web site design and Yahoo!'s business express program are now the two most important factors in determining web site ranking and visibility."

Furthermore, Marckini believes Yahoo's changes represent an "industry mega-trend." "Namely, search databases have now effectively replaced human-edited directories as the perceived most successful search mechanism, as all major search properties are now leading with search databases and subordinating their human-edited directory matches to them," said Marckini.

These experts make two clear points: That Yahoo! is essential because it enhances link popularity, and also that Web page design, and thus search engine optimization, has become even more important as the major portals move toward spider/crawler-driven listings over human-edited listings.

On the Other Hand…

I've seen a few experts question the value of the Yahoo! listing. For instance, software engineer Tom McCracken stated on Sitepoint, "This means that a listing in Yahoo!’s directory will not help your search ranking. Thus there is only one minor advantage to a Yahoo listing, which is that some people do still search for sites using the Yahoo! directory listing (though these users are few and far between)."

Another interesting position is that of Sumantra Roy, in answer to a user question: "Does it still make sense to pay Yahoo! the $299 it requires for getting listed?" Sumantra's answer: “If Yahoo! is forced to modify its search results again and start giving some sort of preference to web sites listed in its directory. At that point of time, it may again make sense to pay the $299 to get listed.”

These views could be short-sighted – time will tell. However, I still believe a Yahoo! listing is essential for commercial sites. It will do a lot more than provide visibility in Yahoo!'s directory listings. I agree with Sullivan and Marckini that it's more important than ever to focus on professional search engine optimization techniques to achieve high rankings in crawler databases.

Value of SEO Over PPC

This statement from Marckini is telling: “Pay-for-placement engines can only get you so far, and the costs are prohibitively expensive with keyword bids up to as much as $5 or $10 per click, some even $50 per click. Monthly keyword budgets in some vertical niches are running upwards of $50,000 to $75,000 per month-advertising dollars that would not have to be spent if the marketers focused on making their Web sites search engine friendly."

Amen bro!

Understanding Search Engine Marketing

It's amazing how many people at the highest levels of management simply do not understand search engine marketing. The single most important issues to consider in 2003: Can the major spider engines see your website? Does your navigation allow the engine to crawl your website thoroughly? Can the engine read your content? Is your content attractive to the engine and your visitors? Do you have killer title and description tags optimized for keywords? Do you have quality sites linking to you?

Google Rules

You must ensure a quality Google listing. In a conversation with Google software engineer Matt Cutts, he reiterated that one step toward high Google rankings is to list in directories first, both DMOZ and Yahoo!.

DMOZ (Open Directory Project) is particularly desirable because it's used by many smaller directories, creating more links. Unfortunately, it can be difficult to get listed if there is no category editor or if an editor is prejudicial. (You can appeal through a "feedback" link that goes to DMOZ staff.)

Site architecture, page design, and optimization of relevant content with strategic keywords in text, headers and tags is of utmost importance, as is a strategic linking strategy.

Also be aware that Google seems to be excluding some cross-linked pages for PageRank ratings. Its spiders now cross check link text with the linked site, excluding links with text that doesn't appear within the linked site. This may be the solution to "Google bombing," whereby linkers capriciously manipulate the PageRank of the linked site.


About the Author
Paul J. Bruemmer is the CEO of Web Ignite, a search engine marketing company founded in 1995. Web-Ignite earned a top grade in the Buyers' Guide to Search Engine Optimization Firms and has helped promote over 15,000 Web sites. Client testimonials report traffic increases of 150 to 500 percent.